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Abstract. We obtain general identities for the product of two Schur functions in the case where 
one of the functions is indexed by a rectangular partition, and give their t-analogs using vertex 
operators. We study subspaces forming a filtration for the symmetric function space that lends 
itself to generalizing the theory of Schur functions and also provides a convenient environment for 
studying the Macdonald polynomials. We use our identities to prove that the vertex operators 
leave such subspaces invariant. We finish by showing that these operators act simply on the 
fc-Schur functions, thus leading to a concept of irreducibility for these functions. 



1. Introduction 

Let A be the ring of symmetric functions in the variables xi, X2, ■ ■ ■ , with coefficients in t). 
The Schur functions, sa[^], form a fundamental basis of A, with central roles in fields such as 
representation theory and algebraic geometry. For example, the Schur functions can be identified 
with the characters of irreducible representations of the symmetric group, and their products are 
equivalent to the Fieri formulas for multiplying Schubert varieties in the intersection ring of a 
Grassmannian. Furthermore, the connection coefficients of the Schur function basis with various 
bases such as the homogeneous symmetric functions, the Hall-Littlewood polynomials, and the 
Macdonald polynomials, are positive and have representation theoretic interpretations. In the case 
of the Macdonald polynomials, H\[X;q,t], this expansion takes the form 

Hx[X;q,t] = J2K^.xiq,t)s^[X], K^xiq,t) e N[q,t] , (1.1) 

where Kf^\{q^t) are known as the (7,i-Kostka polynomials. The representation theoretic interpreta- 
tion for these polynomials is given in |§, 

Recent developments ||, ^ have suggested that a certain filtration of A provides a convenient 
environment for the generalization of natural properties held by the Schur functions, and for the 
study of the Macdonald polynomials. This filtration, C h!f^ C ■ • ■ C a|°°'* = A, is given by 

Kf^ ^ C{Hx[X-t]}x,x,<u^ C{Hx[X-q,t\}^.,^^^^ , (1.2) 

where Hx[X\t\ denote the Hall-Littlewood polynomials. Two bases for these spaces are introduced 
in 1^; the fc-split polynomial basis, which is related to a i-generalization of Schur function products 
1^, In], and the /c-Schur function basis, s^'^^[X;t] |||, ^. The latter basis plays a role in A^''^ 
analogous to the one played in A by the Schur functions. For example, work related to the fc-Schur 



functions prompted a /c-analog of partition conjugation, a refinement of formula (1.1) 



where K^J'-^{q,t) are conjectured to be in N[(7,t], and a generalization of the Fieri and Littlewood- 
Richardson rules. 
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Here, we study classical Schur function properties (and their f-analogs) in the context of the 
spaces a[''\ In particular, the expansion of a product of two Schur functions in terms of Schur 
functions is explicitly known. In our case, we examine the expansion of such a product, where one 
Schur function is indexed by a rectangular partition, in terms of certain products of Schur functions. 
We find, for nonncgativc integers a, r and m and partition v with £{v) < r, 

Sg^r+m Sy — ^ ^ ( ^)^^^ ^a^+fj, ^a—fj,'^,... ,a—fj,'^,u J (1-4) 

where the summand vanishes if (a — /i^, ... ,a — fi[) is not a partition. We also generalize this to 

Ba^+rr^ By ^ ^ {-~t)^^^Ba^j^f^Ba-^'^^^^...^a-ti,'^,v , (1-5) 

where Bx is a vertex operator that reduces to s\ when t = \ . 

This result enables us to provide a thorough analysis of the operators B\ in the context of 
our filtration. We derive commutation relations on B\ as well as a number of other identities for 
these operators and the Schur functions. We find that operators (and Schur functions) indexed by 
rectangular partitions, i.e. partitions of the form (^'^+^^^), play a particularly important role in our 
study. For example, these operators leave fundamental subspaces of invariant. 

Results concerning the fc-Schur function basis arise as a consequence of our work with operators 
indexed by rectangular partitions. In the last section, we prove that the action of such an operator 
on a fc-Schur function produces only one fc-Schur function. Namely, for = 1, 2, . . . , /c, 

(1.6) 

where /i is the partition rearrangement of the entries in [l^^^^^) and A, and is a positive power 
of t given explicitly in Theorem |2^. This result has the important consequence of simplifying the 
construction of the fc-Schur functions. In effect, for each fc, there is a subset of fc! fc-Schur functions 
called the irreducible fc-Schur functions, from which all other s"^\X]t] may be constructed by 
successive application of operators indexed by rectangular partitions. That is, 

4''[X;t] ceN, (1.7) 

where s^/f' [X; t] is an irreducible fc-Schur fimction and Ri, . . . , Rj are rectangular partitions. 

Since the Hall-Littlewood polynomials at i = 1 are the homogeneous symmetric functions, /ia[^], 
Aj''-' reduces to the polynomial ring Since Bf; is simply multiplication by the 

Schur function sr when t = 1, relation (1.7) reduces to 

4^=' [X] ^ SI,, [X]sn._ [X]... SR, [X]s^^^ [X] . (1.8) 

It follows that the irreducible fc-Schur functions thus constitute a natural basis for the quotient ring 
A[''^ /2k, where Xfc is the ideal generated by Schur functions indexed by rectangular shapes of the 
type (£'=+1-0. 
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2. Definitions 

2.1. Partitions. Symmetric polynomials are indexed by partitions, sequences of non-negative in- 
tegers A = (Ai, A2, . . . ) with Ai > A2 > . . . . The number of non-zero parts in A is denoted £{\) 
and the degree of A is |A| = Ai + • • • + Xe(^\). We use Al to denote Xe(^\y Vl^^ denotes the set of 
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all partitions of length at most r, and whose first part is not larger than m. In this fashion, Vl^ 
is the set of partitions of length at most r, and whose first part is equal to m. The case 7^<oq will 
be denoted 7"". Finally, for a partition /i = (/xi, . . . of m entries (possibly zero), the reverse 
reading is denoted /i^ = (/im, • ■ ■ , Mi)- 

We use the dominance order on partitions with |A| — where A < /i when Ai + • • • + A^ < 
Ml + ■ ■ ■ + Mi for all i. Given two partitions A and M; A U m stands for the partition rearrangement of 
the parts of A and /i, A ± m stands for (Ai ± , A2 ± ^2 , • • ■ ) ; s-'^d ( A, m) stands for the concatenation 
of A and m- Note that if A < M ^^^d v < uj, then XiJ v < fiU oj. We shall denote by 5n (or simply 5 
when the value of n is clear) the partition (n — 1, n — 2, . . . , 0). 

Any partition A has an associated Ferrers diagram with A^ lattice squares in the i^^ row, from 
the bottom to top. For example, 

A 



(4,2) 



(2.1) 



For each cell s = (j,j) in the diagram of A, let {s) ^ i{s) , a{s) and a'{s) be respectively the number 
of cells in the diagram of A to the south, north, east and west of the cell s. The hook-length of any 
cell in A, is /is(A) = i{s) + a(s) + 1. In the example, ft.(i.2)(4, 2) = 2 + 1 + 1. The main hook-length 
of A, hMW, is the hook-length of the cell s = (1, 1) in the diagram of A. Therefore, ft.M((4, 2)) = 5. 
The conjugate A' of a partition A is defined by the reflection of the Ferrers diagram about the main 
diagonal. For example, the conjugate of (4,2) is 



A' 



(2,2,1,1). 



(2.2) 



A partition A is said to be k-hounded if its first part is not larger than fc, i.e, if Ai < k. We 
associate to any fc-bounded partition A a sequence of partitions, A~*'^, called the k- split of A. A^*"' = 
(A*^^-', A*^^', . . . , A*^*"') is obtained by partitioning A (without rearranging the entries) into partitions 
A« where /lAf(AW) = fc, for all i < r. For example, (3, 2, 2, 2, 1, 1)^^ = ((3), (2, 2), (2, 1), (1)) and 
(3,2,2,2,1,1)^-* ((3,2),(2,2,1),(1)) . Equivalently, the diagram of A is cut horizontally into 
partitions with main hook-length k. 



.(3) 



□ 



and 



(2.3) 



The last partition in the sequence A^*^ may have main hook- length less than k. It is important to 
note that A^'' = (A) when hM{X) < k. 



2.2. Symmetric functions. The power sum pi{xi,X2, ■ ■ ■) is 

K(xi,a;2, . . .) =x\+xl-\ , (2.4) 

and for a partition A = (Ai, A2, . . . ), 

Px{xi,X2, ■■■) ^ P\iixi,X2,. ■ .)px2{xi,X2,. ■■)■■■ ■ (2.5) 

We employ the notation of A-rings, needing only the formal ring of symmetric functions A to act on 
the ring of rational functions in xi, . . . , xn, q, t, with coefficients in R. The action of a power sum 
Pi on a rational function is, by definition. 



Pi 



E/3 dpv'-p ' 



(2.6) 



with Ca,dp G M and u^, vp monomials in xi, . . . , xjv, q, t. Since the power sums form a basis of the 
ring A, any symmetric function has a unique expression in terms of power sums, and (2.6) extends 
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to an action of A on rational functions. In particular /[AT], the action of a symmetric function / on 
the monomial X = xi + ■ ■ ■ + xat, is simply /(xi, . . . , xn)- In the remainder of the article, we will 
always consider the number of variables N to be infinite, unless otherwise specified. 

The monomial symmetric function m\[Xn] is 

mA[X„]= J2 (2.7) 

(T^Sn-^(T{X) distinct 

The complete symmetric function is 

hr[X]= ^ Xi^Xi2 ■ ■ ■ Xi^ , (2.8) 

1< ii < t2 < ■ ■ -^ir 

and h\[X] stands for the homogeneous symmetric function 

h^[X]^h^,[X]hxAX]-- - . (2.9) 
And, the elementary symmetric function eA[A"] is defined 

ex[X] = exi[X] ex^iX] ■ ■ ■ , where er[X] = ^ Xi.Xi^ ■ ■ ■ Xi,^ . (2.10) 

l<il<i2<---<ir 

Although the Schur functions may be characterized in many ways, here it will be convenient to 
use the Jacobi-Trudi determinantal expression: 

sx[X]^det hx,+j-i[X] (2.11) 

where hr[X] = if r < 0. Note, in particular, Sr[X] — hr[X]. 

We recall that the Macdonald scalar product, ( , )q,t, on A eg) Q{q,t) is defined by setting 

1 _ A. 

{p^[X],p^[X]),^t^Sx^zxl[j-^, (2.12) 

1=1 

where for a partition A with mi{X) parts equal to i, we associate the number 

= l™imi!2'"^TO2!--- (2.13) 
If <7 = t, this expression no longer depends on a parameter and is then denoted (, ), satisfying 

{sx[X],s^[X])^6x^.. (2.14) 
The Macdonald integral forms Jx[X;q,t] are uniquely characterized Q by 

(i) (./a, J^),,t = 0, ifA^Ai, (2.15) 

(ii) Jx[X;q,t]^Y,vx^iq,t).s^[X] with vx^iq,t) e Q{q,t) , (2.16) 

^<A 

(iii)«AA(g,i) = n(l-9"''^*'^'^'"')' (2-17) 

seA 

Here, we use a modification of the Macdonald integral forms that is obtained by setting 

Hx [X; q, t] = Jx [X/il -t);q,t]^Yl ^^^^ ^) ^A^t^l ' (^.18) 

with the coefficients Kfj_x{q, t) G t] known as the g, t-Kostka polynomials. When q — Q^Jx[X; q, t] 
reduces to the Hall-Littlewood polynomial, Jx[X; 0, t] — Qx[X; t]. Again, we use a modification; 

Hx[X;t] = Hx[X;0,t] = Qx[X/{l - t);t] = .sx[X] + J2 K^xit) s^[X] , (2.19) 

Ai>A 



with the coefficients Kf^\{t) G N[i\ known as the Kostka-Foulkes polynomials. The Kostka numbers 
Kf^x e N arise in the limit < = 1, as coefficients in the expansion 

hx[X] = Hx[X- 1] - sx[X] + s^,[X] . (2.20) 

The Kostka numbers also appear in the expansion 

= mA[X] + ^i^A^m^[X]. (2.21) 

3. Vertex operators and Schur functions 

The ring of symmetric polynomials over rational functions in the parameter t has shown to be of 
interest in many fields of mathematics and physics. A natural basis for this space is given by the Hall- 
Littlewood polynomials, H\[X]t]] t-analogs of the homogeneous symmetric functions, /ia[^]. Our 
approach to the study of this space employs vertex operators that arise in the recursive construction 
for the Hall-Littlewood polynomials These operators can be defined for £ e Z, by 

oo 

= ^s,+,[X]s4X(i-l)]^, (3.1) 



where for f,g and h arbitrary symmetric hmctions, / is such that on the scalar product (2.14), 

{f^9.h) = {a.Jh). (3.2) 
Note, ■ 1 = if ^ < 0. The operators build the Hall-Littlewood polynomials by 

Hx[X-t] = Bx,Hx,....,xAX-A, for Ai > A2 , (3.3) 
and, since they satisfy the relation 

BmBn — tBnBm -\- tBm+lBn-l — Bn-lBm+l , 771,71 EZ, (3.4) 

their action on A can be computed algebraically. 

The definition for these operators was extended in jl2j to any partition A of length L, by 

Bx^ n (1 -te^,)BAi •••Sa^ , (3.5) 

l<i<j<L 

where acts by 

Sij {Bxi ■ ■ ■ Bx^) = Bxi ■ ■ ■ Bxi+i ■ ■ ■ Bxj-i ■ ■ ■ Bx^ ■ (3.6) 



It is important to note that ( p.l[ ) gives Bg — si when < = 1, and thus ( |3.5| ) reduces to the 
Jacobi-Trudi formula (2.11) for sa[A]. Therefore, we have 

Bx = sx[X], whent^l. (3.7) 

3.1. Identities on Schur functions and vertex operators. Here we derive properties for the 
vertex operators, and consequently for the Schur functions. Our properties reveal information about 
the behavior of Bx and as a by-product, allow us to prove conjectures relating to a filtration of the 
symmetric function space. The main result of this section is an explicit formula for special products 
of two vertex operators. 

Theorem 1. Let a,r aTid m be noTiTiegative iTitegers and v be a partition with i^iy) < r. Then 

Bar+rr^Bu= ^ ( - < ) ' ^ ' r _|_^ _ ) H _ , (3.8) 

where the summand vanishes if (a™ — {jj!)^) is not a partition. 



Since Bx —> s\ when t = 1 by (3.7), an immediate consequence of this resuh is an exphcit formula 



for a product of two Schur functions. 

Corollary 2. Let a,r and m be nonnegative integers and v be a partition with £(iy) < r. Then 



Sa-+^S^= ^ (-l)"''sa-+pSa"'-(M')«,i" (3-9) 

where the summand vanishes if (a™ — [ji')^) is not a partition. 

From Theorem ^ we derive a number of identities for the product of vertex operators. We now 
state and prove these identities, postponing the proof of our theorem to the end of this section. 
We start by giving several properties of the vertex operators _B.„ that arise from the fact that they 



satisfy the same reordering relations as the Schur functions. Namely (see Proposition 3 of |12 ), for 
an integral vector v, 

-E^V ^Vi,...,Vi — i,Vi^i — l,Vi + l,Vi^2,---,Vj. ■ (3.10) 

This reordering relation allows us to prove the following properties: 

Property 3. If v = (vi, . . . , Vr) £ Z*" is such that Vj Vi — j — i for i ^ j , then By = 0. 



Proof. Without loss of generality, assume j > i- Given By, we successively apply ( 3.10 ) to move 
Vj from position j to position i + I; 

I^V ( 1)^ I^Vi,...,Vi,Vj — {j — i—l),Vi-^-i+l,...,Vj^i+l,Vj^i,...,Vr (^■-^-^) 

= (-1)^ * ^Si;i,...,t>i,i;, + l,t)i+i + l,...,i;j_i + l,'Uj + i,...,t)^ , (3-12) 

since Vj — Vi = j — i. Further, switching the entries in position i and i + 1, we obtain 

which implies that By^^,,,^y.^y.+i^y.^-^+i^,,,^y^_-^+i^y.^-^^,,,^y^ = 0. Consequently, By is also null. □ 

Property 4. Let fi and v be partitions of lengths m and r respectively. If there is a non-zero 
partition reordering, iB\, of B^^jj, then Ai = max{/ii, vi — m} and Xm+r — min{/Xm + r, v^}. 



Proof. An element of v in By moved i steps to the left (right) with ( 3.1Q|) , is decreased (increased) 



by i. Therefore, the only possible entries in the first position of any reordering of B^ ^, are 

/li — (i — 1) , i = 1, . . . , m or Vj — {j ^ m — 1) , j = 1, . . . , r . (3-14) 

Assume the first entry in a reordering B\ of B^^^ is not the largest of these, i.e. Ai < max{/xi, vi—m}. 
Since the entries /ii + i for some i > and vi — j for some j < m must occur in B\, and Ai < 
max{/ii, i^i — m} < max{/ii +i, vi — j}, Ai cannot be the largest entry. Therefore A is not a partition 
unless Ai = max{/ii, vi — to}. Similar reasoning applies for the smallest entry Xm+r- D 

The contrapositive of Property then gives the following result: 



Property 5. Let fi and v be partitions of lengths m and r, respectively. //niax{/xi,^i — to} < 
min{/i„i + r, Vy} then B^.i/ = 0. 

These properties of By allow us to give several identities concerning the product of an operator 
Bi, with an operator indexed by partitions of the form {£''~^^~^), I = 1, . . . hereafter referred 
to as A;-rectangles. Our identities are derived from particular cases of Theorem |[ The first is a 
f-commutation relation. 
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Identity 6. Let i, k and £ be nonnegative integers. For £ < i < k, we have 

B^k+i-i Bj^ ^ t^ ^ Bi B(/k+i-£ . (3.15) 

Proof. Theorem |l], with a — £, r = 1, m = k — £, and v = (z), gives 

k-e 

Bgk+i-^ Bi= {-t)^'"^Be+^ = J2(-*yB^+3 • (3.16) 

Since {£''-^ - (P)^,i) = (£''-^-^ , {£-iy,i), it suffices to show = unless j = i ~ £, 

in which case B^k-i (^£_iy-e i = {—iy^^Bgk+i~i by relation ( 3.10 ). When k > i > j + £, positions 
k~£+l and k — i + 1 of {£^~^^^ ^ {i~ 1)-', ?) contain the entries i and £, resp. Since these entries differ 
by i — £, Property ^ gives Bgk-e-j (^£_iy ^ — 0. Similarly, when £ < i < £ + j , the entries in positions 
k — £ + 1 and k — i, resp i and £ ^ I, differ by i — ^ + 1 and again we have B^k-e-j ^£_iy = 0. □ 

Identity 7. Let v he a partition of length r where hM{v) < k and I'l > £ for £ G N. Then 

B,k^.-, B, = Y^i-ty'-^^Bp^., , (3.17) 

i 

where c{i) £ N, and p*-*-* and 7'-*-' are partitions such that p^*' — i>i, pf^ > £, hM^p^^^) — k,j[^^ — £, 
and hMij^^^) < k (equality holds only when hM{v) = k). 

Proof. Let m — vi — £^ r = k + 1 — vi^ and a = £ in Theorem ^ We then have 

Bgk+i-i B^ ^ ^ (-t)l^lB^fc+i-^i_,_^ . (3.18) 

£(M)<fc + l->'l 

If /ii < vi — £ (cquivalently p'^^_( — 0), then the first and vi — £ + 1*'' entries of {£''^^^ — (m')^j ^) 
are — p'^^_£ — £ and vi respectively. Since these entries differ by i^i — £, Property ^ implies that 
i3£i'i-<_(^/)-R,i, = in this case. We thus have, 

Bik+i-iBy= ^ (-t)l^lB^fc+i-,.i_,.^ i3^^i-«_(^,-)H . (3.19) 

«(M)<fc + l-i'l 

If we let {£''+^-''^+p) = p^') then p^'' = pi+£ = vi. Moreover, £{p) < k+l-vi gives that p^*^ > £ and 
/ijv/(p(*)) = i^i + (fc+l — i^i) — 1 = A;. Further, since {£'^^^^ ~{p!)^ ,i>) is the concatenation of two weakly 
decreasing sequences. Property ||implies any non-zero partition reordering, ±_B^(i), of Bgi,i-i_(^^,-^R jj 

has = m&yi{£~ p'^^_£,vi - {vi-£)} = £. Then, hMh'-'^) =£+(vi-£ + r)-l = vi+r~l<k 
since hM{v) =i'i+r — l<k (equality holds only when hM{v) — k). □ 

We now give two identities regarding the product of an operator indexed by a /c-rectangle, 
j-^fc+i-f-j^ with an operator indexed by any partition with main hook-length exactly k. 

Identity 8. Let v he a partition of length r where hM{v) = k, vi> £, and < £■ Then 

B,k^i-i B, = ^(-t)^(')B^(o , (3.20) 

i 

where c{i) € N, and p*^*' and "f^"^^ are partitions where p^*'' = vi, p^' > £, HmIp^^^) — h^ii"^^^^) = k, 
= £, and = Vr- 

Proof. This result follows from formula ( 3.19 ) in Identity ^ with hM{v) = vi + r — 1 = k and 
Vr < £■ Since £{p) <k + l — vi — r implies p,'i < r, Property ^ proves 7^*'' = min{£ — pi + r, Ur} = Vr 
since £ — p'^+r>£ — r + r = £>Vr. □ 



Lemma 9. Let fi, v, and A he partitions, and let A — w[{ii, v) + (5)) — 5 for some permutation w. 
Let ^ be a partition such that 7 7^ /i, and such that (,{p) — ^(7) = n, where both partitions may 
contain zeroes. Then \ ^ a {i^-^ , y) + 5)^ —5 for all permutations a. 

Proof. Assume there exist permutations w and a where A = w[{pL,v)-\-5)^ —5 = a [{'-^ , v) -\- 5)^ — 5 
for ji ^ ^. This imphes that {^jl^v) + 5 = a' ((7,1^) + 5)) for some permutation a'. For all i > n, 
((/J,, ly) + (5)^ = ((7, u) + (5)^, since £(7) = £{^) — n implies 7.; — = 0. Thus, /i + (5„ = a' (7 + (5„), 
or equivalently /i = (t'(7 + 5„) — 5„, for some a' ^ Sn- However, since 7 is the only element that is 
a partition in the set {t(7 + S„) — (5„ : r G S*,!}, we arrive at the contradiction fi = j. □ 



Identity 10. Let v be a partition of length r where hM{v) — k and Vy > £. Then 

Bgk+i-e = tl'^l-'^^B^ Bi,k+i-i . (3.21) 



Proof. For 7 indexing the partition reordering of i3^"'i-<!_(^')H^y, Identity ^ implies that ( 3.19| ) 



holds with 71 = I. Since hM{i^) = + r — 1 = k, we have ^(7'*^) — i>i— £ + r — k + 1 
Also, £{ii) < fc + 1 — j/i = r leads to fi[ < r. Therefore, Vr > £ implies by Property ^ that 
Ik+i-t — min{i! — ^'^ + r, v^} > £ since ^ — /x'j^+r > £ — r + r = £. Thus, from Property |^, 
71 = "fk+i-t = £ imphes that all non-zero equal ±By where 7 = 

It now suffices to show that Bfui-e_^^,'^R ^ = dzB/^f^k+i-i-^ only when fi = {vi — £^ . . . ,Ur — £). Our 



claim will follow since (3.19) then simplifies to 

Bfik+i-i By = i t^^^ By B^k+i-i , (3.22) 

where the sign must be positive since when t = I, this relation becomes Sgk+i-tSy = s^Sf^k+i-e. In 
fact, we only need to show that when fi — {vi —£,... ,h'r—£), there exists some permutation w where 

u;((ri-^ - iy) + S)-S^ {£^+^-^) . (3.23) 

Then by Lemma since there exists no other partition (f^^^^ — (/i')^) such that w(^{£'^^^^ — {fi')^', + 
5) — 5 = {£'^~^^~''), non-zero terms occur only when /i = (i^i — £). 

To prove ( |3.23| ), it is equivalent to show that there exists some permutation w' where 

w'S + {£''+^-^) = (r^-^ - ip'f ', v) + 6) (3.24) 

Since ^ = {vi— £, . . . ,Vr ~ £) ==> v = {^i+£, . . . ,^r+£), and 6 = (fc — 0) — {r + fii — 1, . . . ,0) 
given r ~ k — vi + I (from hMii^) = fc), we must show there is some permutation w' where 

w'S + (f'^'+i-^) = {£- fi'^^ + r + fii - I, . . . ,£ - n[ + r, pa + £ + r - 1, . . . , fir + £) 

= (^fe+i-^) + {-^'^^ + r + A^i - 1, . . . , -A^'i + r, A^i + r - 1, . . . , Mr) . (3.25) 

The last r entries of w = (— +r + fii — l,..., —fi[ + r, /ii + r — 1, . . . , fir) are fii+r — i, i — 1, . . . ,r, 
and the first fii entries arc r — 1 + j — fi'p j = /ii, . . . , 1. Since r > fi'i, a vector of this type is known 
[H to be a permutation of 5r+pi • Thus, there is some w' such that u = w'6, and ( [3.25| ) follows. □ 

3.2. Proof of Theorem We use several lemmas that rely on properties for the Kostka matrix. 
These properties are derived in Appendix ^ for lack of reference. Here, we use to denote the set 
of vectors of length m, with d ones and m — d zeroes. Then E}^ is the set of vectors v = V1+V2 + ■ ■ ■ , 
where vi € E^\ V2 £ E^ , . . . . 

Lemma 11. For a partition A G and any partition v , we have 

J2 B,W+b^.^ = E ^M+fc^-^A",' • (3-26) 

<TeSr-;cr{X) distinct fJ.£T"' 



Proof. Using Formula we have 

mx+b- = '^'y{^+bn= Y ^\+b-^,+fSf,+b- ■ (3.27) 

(TGSr;o-(A+6'-) distinct fJ-eV^ 

With K^^ — K^]^^^^j^^-r , from Formula ^ this gives 

CTGSr;cr(A) distinct /jET"^ 

We formally replace s, by -B,^,^ since S(t(a) and B„(^x),iy both obey the reordering relation ( |3.1C| ). □ 
Lemma 12. for A G 'P"^, we have 

J2 Ba^-E,.^Y. J2 K-^K^xK^'iB,„._pn^,. (3.28) 

Proof. It is known @ that for A e 7"", 

eA = X K^xKui'^m^ and ^ = eA[xi + • ■ • + Xm] • (3.29) 

Equivalently, we thus have 

E = E E ^-A^-'7 E ^"^^^ • (3.30) 

EeE^ <^ 767"" CTGSm;(T(7) distinct 

Formally replacing x* by Ba^-^^ui this implies 

Ba"^~E,v = X X K^xK^i^ X Ba'^-cr(-i),u ■ (3.31) 

EeE^^ <^ 767"" CTeS„i;CT(7) distinct 

Since a™ — 7^ is a partition, we can use Lemma |ll| with A = a™ — 7-'^ and 6 = to obtain 

J2 Ba^~E,u^J2 E ^-Aif.'7 E ^P.-^C-7-p- (3-32) 

Since ^ = if a™ — 7-'' ^ /i, we have a > /ii. Thus, ~ a™ ~ for some p e T'™^ and 

E Ba'^^E.jy = E E E -p^^iy-fi-V" -7«,a" -p« " (^-^S) 

E;e£;4 7e'P" pe'P"^ 

The property then follows from Formula |3^, which gives K^J,_^ii ^m_pR = -^-^p- ^ 

The last lemma needed to derive our expression for the product of Schur functions and operators 
uses methods presented in ^ and [|2j. Here, v £ [n] is a vector with entries from 0, 1, . . . , n. 

Lemma 13. Let v^fi,"/ be any partitions with ((v) — n, £{fi) = r and £(7) = m. Then 

5^ (-0'"Sp,7+/i3.-£= J2 5] (-<)I'ISm+/57-e... (3.34) 

I={ii,... ,im)G[n] EeE^ /=(ii,... ,ir)e[m] EeE^ 

Proof. Identity (20) in |l2| implies that 

77(C/'-, y'")i7(Z") n n 1 - = H{UnH{V^, Z") J] 11 ^ ^ ' (^.35) 

where H{X^) is a formal Laurent series in an ordered set of variables — {xi,... ,xe) with 
coefficients given by operators which act on P S A by 



H{X'^)P[Y] = P 
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i,j ^ »^ i<i<j<e 



Binomial expansion of (l^l) then gives 



Equivalently, by expanding the products, we have 



H{u^,v"^)H{zn E ••• E ' I!"' E 



=H{u^-)H{v"\z-) E ••■ E ^ r^' r E 

It is known [|| that |^;, = BxP[Y]. Therefore, for partitions ^i, 7, u with = r, ^(7) = 

m,l{v) — n, we take the cocfBcient of m'', w'' and z'' in both sides to obtain 

E ■" E (~^)''^ ^''"-^A',7l+'l,---.7™+im E B^^E 

0<li<n 0<;„i<ri EeEii'-'-'''^^ 

= E "' E .Mr + ir E Bl-E,V 

0<li<in 0<lr<m - .!r) 

This completes the proof. □ 

We can now prove Theorem |l| using these lemmas and our properties for the vertex operators. 
Proof of Theorem |l|. With l{v) ~ n <r, letting ^ — (a^) and 7 — (a™) in Lemma |l^, we have 

E E i~'^)^^^^a'',a+h,... ,a+l„^B,y^E = E E (^^) ' ^ ' l r ■ ■ .o+'r- -^a" -B,!/ ■ 

i=(ii,... dm)e[n] EeE^ L = {h,... ,lr)e[m] EeE^ 

(3.37) 

Property § implies By = Ba'-^a+h,... m+i„, = if 1 < /i < r, since Vr+i - Vr+i-i^ = h. Thus By = 
unless h is zero since < /i < n < Given li = 0, Property ^ imphes By = B^r+i^a+h,... .a+i„ = 
if 1 < ^2 <^ since Vr+2 ~ Vr+2-12 ~ h- Thus By = unless I2 is zero since < I2 < n < r. 
Repeating this argument, Ba'\a+ii,... ,a+i„, — unless /i = ^2 = • ■ ■ = = 0, and we have 

E Ba^ ^a''^ Bij^E = Ba^r+m By = E E ^a+'r ^a'"--E,i' • (3.38) 

BfES"'- - •° L={h,... ,lr)e[m] EeE^ 



Moreover, since Ef^ = for any permutation a, ( 3.3S ) can be written as 



B,.+^By= E (-0'^' E 5-(A)+a- E Ba'^-E,.] ■ (3-39) 

AeT'<^ \o-eSr;o-(A) distinct / \EeE^^ / 

We can now use Lemmas |ll| and ^ to obtain 
B,.,„.By= E (-0'" I E ^M+a-i^r^M (E E E K-^'K^xK^,,B,,._pn^ 



E E E E E k^xk^^]k-^'k^',b,--p-.. 

P'^T^<a 767"" /^eT"' 1^ \^'^T^'<m 



(3.40) 
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It happens that we can sum over all A since any term A ^ 7^<m vanishes by the known property 
Kx^, = R-^ =OliX^^^i. That is, if A ^ 7"', R-^ = since ^ i V. Moreover, if Ai > m, K^x = 
if ijji m. But if uJi > m, we have uj' ^ 7"", and thus K^^iy ~ since 7 G V"^. Therefore, 

peV'j.\ 757"" tj.ev UJ \ X ) 

= E E I E i3a"-p«., (3.41) 

since S^fiK^i^ — if^'^. Again, if 7 ^ "P™, we have X"^^ = since p g "PI^j- Thus, we have 

PSPJP^ pS'P'- V 7 / 

= E Vp-B^+g'- Bam_pR „ 

= ^ (3.42) 
with the restriction that the summand vanishes if (a™ — (/i')^) is not a partition. □ 

4. /c-SPLIT POLYNOMIALS 

Recent developments in the study of the symmetric function space have centered around a filtra- 
tion, Aj^"* C Ap'' C • • • C a|°°'' = A, given by the subspaces 

Ai"^ ^ C{Hx[X;t]},.,^^,^ C{Hx[X;q,t]},.,^^, . (4.1) 

This filtration provides a convenient environment for the generalization of the theory of Schur func- 
tions, and for the study of Macdonald polynomials ^. Several new families of polynomials were 
introduced in ||^ as an approach to studying the spaces A^'^''. In this section, we use results from 
Section ^ to prove properties related to one of these families; the /c-split polynomials. 

Definition 14. For a k-bounded partition A, let — (A'^', . . .). The k-split polynomials are 
defined recursively by 

[A;t] = i?;,a,Gj5,,_^(3,,...)[A;t], with G = 1 . (4.2) 

It was shown in |^ that the fc-split polynomials form a basis for A^'^'', and that vertex operators 
indexed by partitions with hook- length not larger than k leave this space invariant. More precisely, 

Property 15. [| //A is a partition with hM{^) < k, then Bx f E A^'^'' for any f G Aj''-*. 

(k) 

Further, it was shown that Bi acts invariantly on subspaces of AJ , defined for integers a < fc by 

Ai"^"'' ^C{Hx[X;t]}a<x,<k- (4.3) 

Note that in the case a < 0, we simply have A^'^'''-' = A^'^'', and that A^"''^'' can also be defined as 

^C{Gx[X;t]}a<x,<k, (4.4) 
since the transition matrix between the two bases is upper triangular |^ . 

Property 16. [| If i is an integer such that i < k, then Bi f G A^^'^^^ C a[''^ for all f G Aj'^'' . 



We start by extending these properties and then discover, more specifically, that there exist 
subspaces of A^"''^-' that are invariant under a special set of the B\ operators. These subspaces are 
defined, for nonnegative integers j < k, 

f7f =£{Gf (4.5) 

Our generalization of Property ^ is 

Property 17. If X is a partition with /im(A) < k, then Bxf G A^^^''^'' for all f £ a|'^''. 



Proof. Definition ( |3.5| ) gives 

2<j<^(A) 2<i<j<^(A) 

= n (l-tei,)i3A,i?3, (4.6) 

2<]<i{\} 

Since eij increases the index of B\-^ , we have 

i=0 

where Oi is a product of Bj's with j < k. Let / e A^'^^ and note that Property |l6| implies Bj f G A^'^'' 
for j < k. Therefore, O, ■ f e A^'^ and again by Property|l|, Bx,+^ ■ f e x[^^+'-''^ c A^^''''^ since 
Ai + i < Ai + ^(A) - 1 = /lAf(A) < fc. □ 

Property 18. If i is an integer such that I < i < k then Bi f ^ jpy, j ^ A^*"''^'*''' . 

Proof. Let / e Af and assume without loss of generality that / — Hx[X; t] — B\-^H^[X; t] 
for A with Ai > i. When Ai = i + 1, we have Bif = BiBi+iH-^[X;t] = Bi+iBiII-^[X;t] by the 

commutation relation (3^). Property th en im plies that Bif = Bi^iBiH^[X;t] G Aj*^^''^-'. In the 



case that fc > Ai > i + 1, again by relation (3^), we have 

BiHx[X;t] — BiB\^ Hj\X\t] — {tB\^Bi + tBi+iBx^-i — -Bai-iSj+i) • . (4.8) 
The three terms in the right hand side are all of the type BaBhH^, where a > i and b < k. Therefore, 
BaBbH^^ G again by Property ^ □ 

We now prove that the subspaces fjj''^ are invariant under the action of operators indexed by 
fc-rectangles, using the following lemma: 

Lemma 19. If \ is a partition with hM{X) = k and Xl > j then B\ f £ ft^^^ for all f G ^'j^'' ■ 

Proof. Letting / = G^j!^\X;t] with /ii = j, we have that (A,/i) is a partition since /ii < Al- If 
fi^'^ = {fi^^\fi^'^\ . . . ), we thus have (A, fj)^'' — (A, . . . ) since hM{X) = k. Therefore, 

Bx f = Bx (S^d, • • • ) . 1 = g[1]^^ [X; t] (4.9) 
(k) 

by definition, and thus B\ f G ■ □ 
We are now prepared to prove the final result of this section. 

(k) (k) 

Theorem 20. If j < k is a nonnegative integer then B^k+i-i f G ^^max(j £) ^ ^ j 
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Proof. We have either (a) ma,x{j,£) = £ ot (b) max{j,£) — j. In case (a), letting A — {£''^^ ^) 
in Lemma |l^, the assertion holds. Thus assume j > £ and let / = Gi'''*[X;t] where i/i = j. If 
v^'' = (zy*^ . . . ) then there are three possibilities: (i) hM{v'^^'') < k, (m) hM{v''^^) — k and 

< £, or {in) hM{v^^^) = k and v^^^ > £, where L = 

In case (i), /imC^^'^') < k imphes i/^'^ = {i^'-^^) and thus / — • 1. Identity then gives 

• 1 = • 1 , (4.10) 

i 

where the partitions p^^^ and 7^*^ are such that 7^*^)"''^ = (^'^^^7^*^) and p[^'^ — j. Therefore, 

•1 = ^(-0^«G;^L^(0)[^;^] ef^f . (4.11) 

i 

For case (u), we have / — B^(i)B^(2) • ■ • 1. Identity ^ states that 

B,<i, = 5I(-0'^'^^pW , (4.12) 

i 

where p^'^ and are such that {p''''\ ...)^'' = (p(*), 7(*), i/^^), . . .) and pf^ = j. Thus 

Finally in case {Hi), first t-commute B^t+i-f with the operators B^j(i), using Identity |l^, until 

Bik+i-i B^(i) ■ ■ ■ B^(m) • • • 1 = t*B^(i) ■ ■ ■ B^(m) Bik+i-e _B^(m+i) • • • 1 , (4-14) 

where * is a power of t, and where i^^™' is such that v^™'' > £, while ^["^^^^ < £, /lAf (i^'^'"+^^) < k or 
^(m+i) ^ ^]-^ggg scenarios, if /Lt~*'^ = (;y(™+^), . . . ) then p satisfies the conditions of cases 

(a), (i) or (m), resp. We thus have 

/ = i?,.+i-.i?,<,„+i, . • • 1 = B,.+^-, G« [X; t] G fl^'^ . (4.15) 

Since max{£, z/^"^"'"^''} < h'i^\ applying -B^(m) to / then produces an element of ^^^L) by Lemma 
Applying Bj^(i) , . . . , Bj^{m-i} by the same argument gives -B^(i) • • • -B^t^n) B^k+i-i B^(,n+i) • ■ -1 G — 



(k) I k 

proves the theorem from ([4.14[). □ 



5. fc-SCHUR FUNCTIONS 

The fc-split polynomials play a crucial role in the generalization of the theory of symmetric 
functions since they are essential for the construction of another family of polynomials called the k- 
Schur functions. The characterization of this family relies on a projection operator that acts linearly 
on a|''\ for nonnegative integers j < k, by 

^ ' lo otherwise 

Definition 21. Let X be a k -bounded partition. The k-Schur functions are defined recursively by, 
[X;t]=r(fi?A,s[^;^^^^^)[X;i], where sjf)[X;t] = l. (5.2) 
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The fc-Schur functions are beheved to play a role in Aj ' that is analogous to the role the Schur 
functions have in A. It was shown that these functions form a basis for Aj*^^ and that they reduce 
to the Schur functions themselves when k ^ oo. That is, s'"x\X;t] = s\[X] when /im(A) < k. 
Several other properties supporting the claim that the fc-Schur functions generalize the theory of 
Schur functions are given in ^. 

A continuation of the study in Section ^ regarding the action of operators indexed by fc-rectangles 
led to an observation that not all of the fc-Schur functions need to be constructed using Definition |2j. 
We found that for each k, there is a subset of slf''[Ar;i] called the irreducible fc-Schur functions, 
from which all other /c-Schur functions may be constructed by simply applying a succession of 
operators indexed by /c-rectangles. This subset consists of the special set of fc-Schur functions 
indexed by irreducible partitions; fc-bounded partitions with no more than i parts equal to k — i, for 
i = 0, ...,fc-l. 

Definition 22. A k-Schur function indexed by an irreducible partition is said to be a irreducible. 
Otherwise, the k- Schur function is called reducible. 

For example, the irreducible fc-Schur functions for A; = 1, 2, 3 are 
k = 1 : \ 
k — 2 . '^0 ' ' 

_ o . J3) J3) (3) (3) (3) (3) 

ft, — O . ''o ' ''I ' ■'2 ' I ''2,1 ' 2,1,1 • K^-"^) 

These examples support the following property; 

Property 23. [|| There are kl distinct k-irreducible partitions. 

The main result of this section is to prove that an operator indexed by a /c-rectangle R sends, 
up to a constant, s^'^''[A';t] to s^^^_,^[Ar; i]. Consequently, given the fc! irreducible fc-Schur functions, 
any reducible fc-Schur function can be obtained by applying a sequence of operators indexed by 
fc-rectangles to the appropriate irreducible fc-Schur function. We first give several properties of the 
operators B\ and Tj''^ and then, using our results from Section ^, we will prove our main result. 

Property 24. [| If X is a partition with hiviW < k then t'^^ B^f = T^^^^Bx^B-J for all f e A^ \ 



Property 25. If k> j>i are nonnegative integers, Tj''^ B^k+i-i f — B^k+i-iTj''^ f for all f e Aj'^^ 

Proof. It suffices to consider / — G^*^'' [A"; t]. By the definition of Tj''\ we have 

B,...-.Tf)/=|° ^ij^^^^ (5.4) 

[Bek+i-t f if j = Ai 

On the other hand, consider Tj''^ Bfk+i-e f . By Theorem B(k+i-t f G f^|^ax{f Ai}- J — Ai > 
then B^k+i-e f e nf^ => T^'"'^ Bf-k+i-i f = B^k+i-i f. If j ^ Ai then either B^k+i-e f e flf^. or 
Bf^k+i-e f G ^'i.^^j- Both cases vanish under the action of Tj*^-* and thus we prove our claim. □ 
We can now show that an operator indexed by a fc-rectangle acts simply on a fc-Schur function. 

Theorem 26. // /i, v, A are partitions where A ~ (/i, i^) and /ii > £ > i^i, then 

B,k^.^. sf\X;t] = il^l-^('^)^sJi+,_.)^,[X;t] . (5.5) 



Proof. Let = M. Since Am = Mi > ^ ^-^d -^Af+i = < ^^ we have 

4'=)[X;<] = i^i+-+^--*«Tif i?A,Tif . . .Ti;;]SA„i?,.+i-. 5^1^; , (5.6) 
by iterating the foUowing argument M times: If A = (A2, . . . , A^), by definition of s^^^ we have 

Buk+i-t sf^ [X; t] = B^k+i-t s^^^ [X; t] . (5.7) 

Since sf' € \ Property |l| gives that Bx.sf^ e \ Property |2| then imphes that T^^^ 
commutes with B^k-t+i since Ai > £. We thus have 

Furthermore, Bgk-i+i f-commutes with B\^ by Identity |^. Therefore, 

Bgk+i-i sf\X;t] = t^^-^T'^^Bx,Bik+i-i s^^\x-t\ . (5.9) 

Now if we can show, for a partition v with vi < £, that 

Bgk+i-i = frf ^ BeY^' ' , (5.10) 



then putting this into (5^) imphes our result by definition, since Am > ^ > Am+i- To prove ( |3.10 ), 
we have sl.'^^ G fl^lf' by definition, which imphes B(k+i-es^^ G ftg^'^ by Theorem ^ since < £. 

(k) . . . (k) /P K 

Therefore, Bgk+i-tSu is invariant under and ( p. 10 ) is equivalent to 

/ \ k-\-l — i 

rf ^B^.+i-.4''^[^;*] = (rf 4''^[^;^]- (5-ii) 

Using Property the right hand side may be written 

^ 4''^[^;^] - 4'='[X;t] . (5.12) 

Now, Bf,k-is';^\X;t] e A^"'"' by Property Further, since any element of Aj' ' can be decomposed 
into the sum of two functions, / G flf ^ and g G A^^"''^''^'', we have Bgk-t s'il'^X; t] = / + .g. Therefore, 

Tf ^ Be if + g) = T^^ Be rf ^ / = Tf ^ Tf ^ (/ + g) (5.13) 

since • .g G Af+^'''^ by Property |l| implies t/''^B^ -5 = 0. Thus, ( ^.12[ ) gives 

Tf 4'=)[X;t] = T^''^BeT^''^B,k-i 4'=)[X;i] . (5.14) 

Repeating this argument fc — £ times proves ( ^.10 ). □ 

When t — I, our theorem reduces to the simple expression: 

Corollary 27. If X is a k -bounded partition, then 

Sek+^-e[X] si'\x] = sJ4,_,)^,[X] . (5.15) 

The role of Schur functions indexed by fc-rectangles in the subring A*^'') leads naturally to the study 
of the quotient ring A'^'^^^^fe; where Xj, denotes the ideal generated by S(^ek+i-e-^[X]. It is known that 
the dimension of this quotient ring is k\ since 

Proposition 28. |^ The homogeneous functions indexed by k-irreducihle partitions form a basis of 
the quotient ring Af'^^/Xfe. 

Corollary then implies 

Theorem 29. The irreducible k -Schur functions form a basis of the quotient ring K^'^^ /X^. 
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The irreducible /c-Schur function basis thus offers a simple method of performing operations in 
this quotient ring: first work in A*^*^') using fc-Schur functions and then replace by zero all the fc-Schur 
functions indexed by partitions which are not fc-irreducible. 



6. Appendix 



Formula 30. Let /i, A G P™. The Kostka matrix is such that 



(6.1) 



Proof. We have 



mx+a"^ [xi H h x^] = {xi ■ ■ ■ XmT mxlxi A h x^] 

= (xi • • • x„X 51 -^A^^ Sf,[xi + --- 

= 5^ S/i+a'" [xi + ■ ■ ■ + a;„i] , 
since (xi • • • x„i)"- s^^ [xi H h x^] = s^+a™ [xi ^ h x„i]. By definition 

mx+a"- [Xi H h X„] ^ ^A+a",i/ s^[xi^ h x„ 



Taking the coefficient of s^+a™ in (6^) and (|6.3|), we get -fC;^^^ = -^A+a™ ^i+a™' claimed. 



Formula 31. Let X be a partition of at most m parts, and let a > Ai. Then 

'I I' 
\ \ 



(Xi • ■ -XmT = Sa^-XR[X1 H h X^] • 



(6.2) 

(6.3) 

□ 

(6.4) 



Proof. In § {Sf5), one finds 
1 1 



Sx 



Xl 



S{a"-/X')'[xi H l-x™]/si™[xi H hXm]'' 

S(a™/A')' [2^1-1 ^ X-rn]/ (xi ■ ■ ■ XmT ■ 



Following from the Jacobi-Trudi determinantal expression for skew Schur functions, we have 



= det 



Sfii-Xj-i+j]^ 



— det s 



M™ + l-j-A,.+ l-i-(n+l-i) + (n+l— ll<ij-<m 
l<i,j<m 



Therefore, if /i = (a™), we obtain 



Sa'"/A - det |sa„A„+i_.-i+i|i<,^ ■<„ - 



(6.5) 



(6.6) 

(6.7) 

□ 



completing the proof. 

Formula 32. Let /i, A £ V"^ , and let a be an integer such that a > Ai and a > /ii. Then, the Rostka 
matrix is such that 



Proof. Using Formula we have 

mam_xR [xi H h x„] = (xi • ■ • XmT "^A 



(6.8) 



{x,---XmTY.^^>^ 

^R^l Sa,r._^R [Xl + 



1 

h h - 

Xl .X. 

' f 

Xl 



1 



(6.9) 
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Since, by definition, 

m<j,r._;^fl[xi H h Xrn] = ^ ^a"^-\R,i, [xi ~\ h Xm] , (6.10) 



taking the coefRcient of Sam_u« in Q and we get if;;"^^ = K-J,_ 

-\R as claimed. □ 



Formula 33. If X = (Ai, . . . , A„), then 

E E MA) (6.11) 

(TG5n,;cr(A) distinct (TGSn;o'{X) distinct 

Proof. The formula holds if and only if 

E ^^^'^ = E '^W ' (6.12) 

since summing over all elements of Sn adds the same symmetry factor on each side of the equation. 
If we insert 



«-(A) = ^^ ITTITTiWA^' (6-13) 



where e(w) is the sign of the permutation w, we get 

E eiw)x-^^'>+^'^'^ = E eH^'"^"^^^+'^- (6.14) 

Now, letting wa = o"' in the RHS of this equation, we obtain 

which proves the formula. □ 
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